Papers with average loss
Seq2Path: Generating Sentiment Tuples as Paths of a Tree (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing generative methods for extracting sentiment tuples do not have orders between the t-uples . a novel parallel generative framework for ABSA is proposed . |
| Approach: | They propose a parallel generative framework to generate sentiment tuples as paths of a tree . they train the model with an independent target and introduce a discriminative token . |
| Outcome: | The proposed method achieves state-of-the-art on AOPE, ASTE, TASD, UABSA, ACOS . it trains with the loss of ordinary Seq2Seq averaged over paths, and inferences automatically select valid paths. |
MoPe: Model Perturbation based Privacy Attacks on Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that Large Language Models can unintentionally leak sensitive information . a new method to identify with high confidence if a given text is in training data is proposed . |
| Approach: | They propose a method to detect if a given text is in a pre-trained language model . they show that MoPe is more effective than existing loss-based attacks . |
| Outcome: | The proposed method is more effective than loss-based attacks and perturbation-based methods. |
Measuring Distribution Shift in User Prompts and Its Effects on LLM Performance (2026.acl-long)
Copied to clipboard
| Challenge: | a large-scale evaluation of deployed LLMs under natural prompt distribution shift is needed . natural prompt behavior shifts can cause performance degradation in dynamic, real-world settings . |
| Approach: | They propose a data-centric framework for measuring natural prompt distribution shift . they train models on 4.68M training prompts and evaluate on 57.6k prompts . |
| Outcome: | The proposed framework evaluates natural prompt distribution shift in LLMs over time and between user groups. |